Learning to Obtain Reward, but Not Avoid Punishment, Is Affected by Presence of PTSD Symptoms in Male Veterans: Empirical Data and Computational Model

Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD

Congenital Insensitivity to Pain: Novel SCN9A Missense and In-Frame Deletion Mutations

SCN9A encodes the voltage-gated sodium channel Nav1.7, a protein highly expressed in pain-sensing neurons. Mutations in SCN9A cause three human pain disorders: bi-allelic loss of function mutations result in Channelopathy-associated Insensitivity to Pain (CIP), whereas activating mutations cause severe episodic pain in Paroxysmal Extreme Pain Disorder (PEPD) and Primary Erythermalgia (PE). To date, all mutations in SCN9A that cause a complete inability to experience pain are protein truncating and presumably lead to no protein being produced. Here, we describe the identification and functional characterization of two novel non-truncating mutations in families with CIP: a homozygously-inherited missense mutation found in a consanguineous Israeli Bedouin family (Nav1.7-R896Q) and a five amino acid in-frame deletion found in a sporadic compound heterozygote (Nav1.7-ΔR1370-L1374). Both of these mutations map to the pore region of the Nav1.7 sodium channel. Using transient transfection of PC12 cells we found a significant reduction in membrane localization of the mutant protein compared to the wild type. Furthermore, voltage clamp experiments of mutant-transfected HEK293 cells show a complete loss of function of the sodium channel, consistent with the absence of pain phenotype. In summary, this study has identified critical amino acids needed for the normal subcellular localization and function of Nav1.7. © 2010 Wiley-Liss, Inc

The Role of Informative and Ambiguous Feedback in Avoidance Behavior: Empirical and Computational Findings

<div><p>Avoidance behavior is a critical component of many psychiatric disorders, and as such, it is important to understand how avoidance behavior arises, and whether it can be modified. In this study, we used empirical and computational methods to assess the role of informational feedback and ambiguous outcome in avoidance behavior. We adapted a computer-based probabilistic classification learning task, which includes positive, negative and no-feedback outcomes; the latter outcome is ambiguous as it might signal either a successful outcome (missed punishment) or a failure (missed reward). Prior work with this task suggested that most healthy subjects viewed the no-feedback outcome as strongly positive. Interestingly, in a later version of the classification task, when healthy subjects were allowed to opt out of (i.e. avoid) responding, some subjects (“avoiders”) reliably avoided trials where there was a risk of punishment, but other subjects (“non-avoiders”) never made any avoidance responses at all. One possible interpretation is that the “non-avoiders” valued the no-feedback outcome so positively on punishment-based trials that they had little incentive to avoid. Another possible interpretation is that the outcome of an avoided trial is unspecified and that lack of information is aversive, decreasing subjects’ tendency to avoid. To examine these ideas, we here tested healthy young adults on versions of the task where avoidance responses either did or did not generate informational feedback about the optimal response. Results showed that provision of informational feedback decreased avoidance responses and also decreased categorization performance, without significantly affecting the percentage of subjects classified as “avoiders.” To better understand these results, we used a modified Q-learning model to fit individual subject data. Simulation results suggest that subjects in the feedback condition adjusted their behavior faster following better-than-expected outcomes, compared to subjects in the no-feedback condition. Additionally, in both task conditions, “avoiders” adjusted their behavior faster following worse-than-expected outcomes, and treated the ambiguous no-feedback outcome as less rewarding, compared to non-avoiders. Together, results shed light on the important role of ambiguous and informative feedback in avoidance behavior.</p></div

Empirical data from the no-feedback condition (green bars), in which subjects could avoid categorization by “skipping” the trial, and the feedback condition (orange bars) in which avoidance responses generated informational feedback.

<p>Subjects in the no-feedback condition made more optimal classification responses (A) and avoided (skipped) more punishment-based trials (B) than subjects in the feedback condition. When these measures were combined into an overall measure of adaptive responses (optimal classification plus avoidance of punishment-based trials), subjects the no-feedback condition showed better performance on both reward-based and punishment-based trials (C), resulting in significantly more total points accrued by subjects in the no-feedback condition (D). In this and subsequent figures, error bars represent SEM.</p

Example screen events.

<p>(A) On each trial, the participant sees a stimulus and is asked to classify the stimulus as belonging to category “A” or “B,” or to “skip”(avoid) the trial. If the subject makes a classification response, the chosen category is then circled. (B) For reward-based stimuli, correct responses are rewarded with feedback and point gain, while (C) incorrect responses receive no feedback. (D) For punishment-based stimuli, incorrect responses are punished with feedback and point loss; incorrect responses receive no feedback (similar to C). If the subject makes an avoidance response, the computer simply acknowledges the response and moves on to the next trial.</p

Correlations between avoidance behavior and estimated parameter values (A) <i>LR+</i>, (B) <i>LR-</i>, (C) <i>T</i>, (D) <i>R0</i>.

<p><i>LR</i>- was positively correlated with avoidance responses, indicating that subjects who made more avoidance responses tended to learn more quickly from worse-than-expected feedback; <i>R0</i> was negatively correlated with avoidance responses, indicating that subjects who made more avoidance responses tended to more negatively value the ambiguous neutral feedback on categorization trials.</p

Computational models of post-traumatic stress disorder (PTSD)

Post-traumatic stress disorder (PTSD) is a multi-faceted, often chronic syndrome that may develop following an exposure to a highly-traumatic event. Given the complexity, heterogeneity, and incomplete theoretical description of this disorder, computational models of PTSD are an important tool to help understand the mechanisms, and course, of symptom presentation and maintenance in PTSD. This chapter reviews several computational modeling approaches and their implications for PTSD, including models focused on fear learning and expression, changes in arousal and reactivity, avoidance, changes in cognition and mood and intrusive recollection. It also discusses the limitations of each approach, and suggests possible directions for future research that could both advance our understanding of PTSD and help move toward a comprehensive computational account of this disorder

Empirical data from the no-feedback condition (green bars), in which subjects could avoid categorization by “skipping” the trial, and the feedback condition (orange bars) in which avoidance responses generated informational feedback.

<p>Subjects in the no-feedback condition made more optimal classification responses (A) and avoided (skipped) more punishment-based trials (B) than subjects in the feedback condition. When these measures were combined into an overall measure of adaptive responses (optimal classification plus avoidance of punishment-based trials), subjects the no-feedback condition showed better performance on both reward-based and punishment-based trials (C), resulting in significantly more total points accrued by subjects in the no-feedback condition (D). In this and subsequent figures, error bars represent SEM.</p

Reaction time (RT) measures.

<p>(A) On trials where a classification response was made, there was generally slower responding on punishment than reward trials and on trials where the non-optimal response was made; there was also a main effect of task condition, with subjects in the no-feedback condition generally responding faster than subjects in the feedback condition. (B) On trials where an avoidance response was made, there were no significant differences between reward-based and punishment-based trials, but subjects in the no-feedback condition again responded faster overall than subjects in the feedback condition.</p

Correlations between avoidance behavior and estimated parameter values (A) <i>RSrew</i> and (B) <i>RSpun</i>, for subjects who made at least one avoidance response to the corresponding trial type, and whom these parameters therefore had defined values.

<p>Both <i>RSrew</i> and <i>RSpun</i> were positively correlated with avoidance responses on the corresponding trial types, indicating that subjects who made more avoidance responses tended to more positively value the resulting feedback.</p